1. Field of the Invention
This invention relates to imaging systems in which specifically aberrated optics are balanced by specific digital image processing.
2. Description of the Related Art
Electro-optic imaging systems typically include imaging optics (e.g., a lens or mirror assembly), an electronic sensor array (e.g., CCD detector array) and a digital image processor (e.g., typically implemented in dedicated chips or software). Traditional methods for designing these systems generally involve fairly independent steps. The optics typically is designed with the goal of forming a high quality intermediate optical image at the sensor array. The sensor array often is dictated by the intended application, including resolution and cost factors. The digital image processing typically is designed after the optics, with the goal of compensating for remaining defects in the sampled intermediate optical image.
The design stages typically occur with very little coordination between the optical designer and the image processing designer. The separation of these stages is a reflection of the significant differences between the fields of optics and image processing in their methods, tools, goals and constraints. For example, each field covers a large swath of potential applications but there typically is little overlap between the two fields other than the design of electro-optic imaging systems. The design of conventional microscopes, telescopes, eyeglasses, etc. typically does not consider any significant image processing. Likewise, areas of image processing such as compression, computer graphics, and image enhancement typically do not involve any significant optics. As a result, each field has evolved independent of the other and with its own unique terminology, best practices, and set of tools. In general, the familiarity required to master each of these domains hinders a unified perspective to designing electro-optic imaging systems.
One drawback to the traditional design approach is that synergies between the optics and the digital image processing subsystem may be overlooked. The optical designer creates the “best” optical subsystem without knowledge of the digital image processing subsystem. The image processing designer creates the “best” digital image processing subsystem without the ability to modify the previously designed optical subsystem. These subsystems are then “glued” together to form the electro-optic imaging system. The concatenation of two independently designed “best” subsystems may not yield the “best” overall system. There may be unwanted interactions between the two independently designed subsystems and potential synergies between the two subsystems may go unrealized.
There has been recent interest in taking advantage of these synergies. For example, U.S. patent application Ser. No. 11/155,870 “End-To-End Design of Electro-Optic Imaging Systems” to Robinson and Stork concerns a general approach to designing an imaging system by allowing the imaging optics and image processing to compensate each other. Thus, while neither the optics nor the image processing may be optimal when considered alone, the interaction of the two produces good results. Put in another way, in order to achieve a certain overall image quality, this approach allows the use of lower quality optics and/or lower quality image processing so long as the two compensate each other to achieve the desired performance.
However, this is a design problem with a very large degree of freedom. In the traditional approach, the separate design of the optics alone is a difficult problem with a large possible design space and it takes professional optical designers many years to master the task of selecting good designs within this design space. Similarly, the subsequent design of the image processing is also a large possible design space and it also takes professionals in that field many years to master their art. In the combined approach, the optics and image processing are designed together, thus the total possible design space includes both the optics design space and the image processing design space, thus compounding the complexity of the design problem.
As a result, the designs to date that are based on some interplay between the optics and the image processing typically are either fairly specific designs or designs that take advantage of a narrow characteristic (i.e., not exploring the entire design space). For example, U.S. patent application Ser. No. 12/215,742 “Electro-Optic Imaging System With Aberrated Triplet Lens Compensated By Digital Image Processing” to Robinson and Stork presents various designs, most if not all of which are specific triplet designs. U.S. patent application Ser. No. 11/768,009 “Compact Super Wide-Angle Imaging System” to Robinson presents various designs, most if not all of which are targeted for fisheye imaging applications. U.S. patent application Ser. No. 11/866,860 “Catadioptric Imaging System” to Robinson presents various designs, but for a specific class of catadioptric system.
Similarly, there have also been attempts to trade off or take advantage of specific optical characteristics. For example, U.S. application Ser. No. 11/999,101 “End-to-End Design Of Electro-Optic Imaging Systems For Color-Correlated Objects” to Robinson takes advantage of color correlation when imaging color-correlated objects. U.S. Pat. No. 7,224,540 “Extended depth of field imaging system using chromatic aberration” to Olmstead et al. and U.S. Pat. No. 5,468,950 “Chromatic ranging method and apparatus for reading optically readable information over a substantial range of distances” to Hanson also take advantage of color correlation, but for objects where different color images are perfectly correlated. U.S. Pat. No. 5,748,371 “Extended depth of field optical systems” to Cathey, Jr. et al. uses a different approach based on introducing a phase mask, but also for color-correlated objects.
Another class of designs concerns extending the depth of field for an imaging system by introducing other types of aberrations or optical effects that, when acting in conjunction with subsequent image processing, will compensate for depth of field. For example, Dowski and Cathey, “Extended Depth of Field through Wavefront Coding,” Applied Optics 41:1859-1866, 1995, inserts an additional phase plate in order to increase the depth of field. As another example, George and Chi, “Extended Depth of Field Using a Logarithmic Asphere,” Journal of Optics A: Pure Applied Optics 5(5):S157-S163, 2003, uses a special aspheric element to increase the depth of field. U.S. Pat. No. 7,336,430 “Extended Depth of Field Using a Multi-Focal Length Lens with a Controlled Range of Spherical Aberration and a Centrally Obscured Aperture” to George and Chi introduces a central obscuration and spherical aberration to increase the depth of field. Mouroulis, “Depth of Field Extension with Spherical Optics,” Optics Express 16(17):12995-13004, 2008, concerns extending the depth of field by using spherical aberration.
Most, if not all, of the above designs are either specific designs or dependent upon a specific approach or optical characteristic. In addition, many of the above approaches require the use of expensive optical elements (e.g., the introduction of additional phase plates, pupil masks, or other elements or the use of aspheres or other more complex optics) and/or complicated image processing (e.g., iterative deconvolution or ideal Wiener filters).
Thus, there is a need for approaches that are broader in nature (i.e., potentially covering a wider range of applications and imaging systems). These approaches preferably use simple optics and simple digital filtering, while also simplifying the design task (i.e., limiting the design space).